The present invention relates generally to a simulation method and system for testing computer networks, and more particularly to a method and system for generating local area network (xe2x80x9cLANxe2x80x9d) traffic for multiple simulated client workstations.
Computers and computer networks currently provide important advantages to enterprises and individuals in today""s society. Moreover, with the advent and ensuing popularity of the Internet and the World Wide Web (xe2x80x9cWebxe2x80x9d), there is a tremendous increase in volume and usage of networked computer systems. Consequently, the explosive growth of computer networks has necessitated a need for larger servers to handle the network traffic. Presently, different implementation paradigms are available for handling this increase in volume. For example, some developers in the computer industry are implementing larger single servers to handle the network traffic while others are including a greater number of duplicated, relatively small servers to handle the increase in the network traffic volume.
Typically, as part of a development cycle, computer systems and software applications are tested by utilizing a simulation test tool, designed to emulate a real run-time environment in order to test whether the specific computer system or application meets the various design criteria. In both of the above-described server implementations for meeting the high demand of network e) traffic, a complete and thorough test requires that the servicing computer, e.g., a server comprising a single machine or multiple machines, be tested at full load. Failure to execute a complete test inevitably results in a failure in the system when a full load is eventually impressed upon the server during the run-time. Accordingly, to effectively test large and complex distributed applications and/or server applications, simulation of a realistically large client base is needed. Currently existing simulation tools provide simulation at application level. However, simulation at the application level yields an application specific simulator. Simulation at the port level is an improvement but still fails to exercise many client-specific paths in the system under test. Thus, a general purpose simulator that seeks to provide a high fidelity simulation, for example, a simulation at level 2, the data link layer of the protocol stack, is highly desirable. The data link layer is the lowest protocol stack level where each individual client has a unique client address.
Other simulators are currently available for testing computer networks to withstand a large volume of traffic. FIG. 1 is a diagram 100 illustrating a typical prior art simulation setup. The serving system, i.e., a system under test 102 is driven by a set of simulation driving systems, S 104. These driving systems 104 are controlled by a central machine, simulation controller 106. All of these machines share a common connection medium 108 such as a local area network or LAN. A plurality of such configurations may be present in order to achieve the total bandwidth necessary to drive the serving system. There may be multiple controlling systems or the controlling system may have connections to multiple LANs.
Typically, the existing prior art simulators fall into two broad categories: 1) keystroke stuffers; and 2) protocol exercisers. A xe2x80x9ckeystroke stufferxe2x80x9d utilizes the actual application on the driving client. The driving vehicle is a program which retrieves stored scripts of keystrokes, mouse movements, and/or other user inputs to provide input to the actual client application to drive the application. The application then directs the traffic through a client protocol stack to the system under test 102.
FIG. 2 is a diagram of the seven layer Open Systems Interconnection (xe2x80x9cOSIxe2x80x9d) protocol stack model. Each of the layers represents a function that must be performed to effect communication between different machines. The lowest layer in the model is the physical layer 202. The functions within the physical layer 202 include setting up, maintaining, and deactivating physical circuits U) between systems. The most notable physical layer interfaces include IEEE 802.2 and IEEE 802.3. The next layer, i.e., layer 2, in the OSI model is the data link layer 204, which is responsible for transferring data over the physical circuits or the channel between systems. The functions of the data link layer 204 include dividing data into frames in order to transfer the frames to another system across the physical medium. The data link layer provides for the synchronization of data to delimit the flow of bits from the physical layer. The data link is a point-to-point link between two devices that are directly connected together.
The next layer, layer 3, is the network layer 206. The network layer 206 provides inter-network services such as the network routing and the communications between networks. The network layer 206 handles multiple point-to-point links in the case where frames are transmitted across multiple links to reach their destination. Internet Protocol (xe2x80x9cIPxe2x80x9d) in the Transmission Control Protocol/Internet Protocol (xe2x80x9cTCP/IPxe2x80x9d) suite is a network layer protocol.
The next layer, layer 4, is the transport layer 208. The transport layer 208 provides end-to-end accountability of data transmitted as streams of packets. The transport layer functions include monitoring data flow to ensure proper delivery of data between source and destination. It provides for error correction and for data fragmentation and reassembly. TCP is a protocol layer protocol.
The next layer up, layer 5, is the session layer 210. The session layer 210 provides for control and synchronization in exchanging data between users. For example, dialogues may be used for check and recovery of data transfer. The next layer up, layer 6, is the presentation layer 212. The presentation layer 212 functions include formatting data for display or presentation. In this layer, codes and encryption in data are interpreted and formatted for presentation. The next layer up, layer 7, is the application layer 214. This layer is responsible for supporting end-user applications such as file transfer, electronic message exchanges and terminal session.
Importantly, it should be understood that all layers of the above-described OSI model are not necessarily present in all protocol stacks. That is, the OSI protocol stack is a reference model to provide standardized logical decomposition of network into layers for communications between systems. For example TCP/IP does not have a complete level 4 in the protocol stack because IP does not guarantee delivery of data nor will it detect missing packets, although there is a reassembly function in TCP/IP that serves a portion of the function typically found at level 4. Instead, for TCP/IP, reliable delivery is left up to the driving application at level 7. Thus, a keystroke stuffer is, effectively, simulation above level 7, the application layer of the protocol stack.
The second type of prior art simulators is a xe2x80x9cprotocol exerciser,xe2x80x9d which provides a somewhat more efficient method for producing load. This method involves simulation at level 7 of the protocol stack. This type of simulator will have knowledge of a particular application, for example, the File Transfer Protocol (xe2x80x9cFTPxe2x80x9d). This simulator will open ports directly with the system under test rather than having the client application do so and drive traffic to the system under test by conforming to the higher level protocol that is being tested, e.g., FTP, Hypertext Transfer Protocol (xe2x80x9cHTTPxe2x80x9d), Open Database Connectivity (xe2x80x9cODBCxe2x80x9d), Simple Mail Transfer Protocol (xe2x80x9cSMTPxe2x80x9d), etc. The protocol exerciser provides an efficient way to generate load because all extraneous functions of the application such as redrawing screens, etc., do not need to be performed at the client. A significant drawback of this type of load generation, however, is that this simulator is protocol specific. If a new protocol is developed, a new load generator needs to be developed. For example, an SMTP traffic generator cannot be used to test the Internet Mail Access Protocol (xe2x80x9cIMAPxe2x80x9d) even though both protocols are mail transfer protocols. Instead, it is necessary to develop an IMAP protocol tester.
The result of either of these methods is a series of packets that pass between the system under test (102 FIG. 1) and the driving system (104 FIG. 1). FIG. 3 is a layout of a packet for a Transmission Control Protocol/Internet Protocol (xe2x80x9cTCP/IPxe2x80x9d) transmission. This packet includes a series of headers 302, 304, 306 that have been prepended to the data 308 that the application desires to send. FIG. 3 also indicates the level 310, 312, 314 in the protocol stack where the data is added to the packet and the address 316, 318, 320 that applies to each level. The application data 308 from layer 7 is typically passed to the transport layer protocol, for example, the TCP. As shown in FIG. 3, TCP adds a header 306, and passes it to next layer protocol 312, for example, the IP. IP also adds its header 304 and passes it to the layer 2 protocol 310 which also adds its header 302. As shown, layer 2 protocol may be Ethernet, Token Ring, FDDI LAN. The padded data is then passed to the physical layer where the data is converted into electrical signals and transmitted to the destination system to be received by the network interface card on the destination system. In sum, a request from a user traverses the OSI model until it is converted into network traffic. Once it reaches the destination system, it moves back up the OSI model so that a server application may interpret the request.
FIG. 4 shows the layout of the LAN header 400. This is an IEEE 802.2 Token Ring header. The shaded xe2x80x9cSource MACIDxe2x80x9d field 402 is the client address which is added at level 2. MACIDs are assigned by the manufacturer of the network interface card, and they are unique to each card. FIG. 5 shows the layout of the IP header 500. The shaded xe2x80x9cSource IP addressxe2x80x9d field 502 is the client address which is added at level 3. FIG. 6 shows the layout of the TCP header. The shaded xe2x80x9cSource Portxe2x80x9d field 602 allows the receiving host to route the data field to a specific application running on that client. The header formats shown in FIGS. 4, 5, and 6 are well known to those skilled in the art of computer networking. Note, that the port numbers are not client unique. That is, given a port number, it is not possible to identify the unique client to which that port number belongs. Instead, port numbers are used to distinguish between various users within the client. In contrast, the MACID and IP addresses are client unique. Given either a MACID or an IP address, a unique client can be determined. Given this overview, it is possible to examine the traffic flows resulting from simulators of the current art, to compare these flows with traffic that would result from an equal number of actual clients, and to compare and contrast how these flows would affect the system under test.
The major difference between a prior art simulation tool and real client traffic is the degree of fidelity of the simulation. Generally, the phrase xe2x80x9ctest fidelityxe2x80x9d refers to a test properly exercises all code paths, for example, to closely emulate the real run-time network load in a computer network. FIG. 7 shows a series of packets 700 that would originate from a prior art simulator where a single workstation was set up to simulate 100 clients. Generally, the prior art simulation method would open one hundred sessions with the system under test. This would result in one hundred packets being sent to the system under test. However, as illustrated in FIG. 7, the client specific addresses, MACID and IP address 702a . . . 702n, 704a . . . 704n, for each of these packets would all be identical. It is apparent to both the system under test and to anyone who might be tracing the traffic of these transactions that the test setup has a single workstation providing the entire volumes of traffic.
In contrast, FIG. 8 shows one hundred packets 800 produced by a real client set. Note that each packet has unique client-specific addresses for both the MACID and the IP address 802a . . . 802n, 804a . . . 804n. Thus, to the system under test, this appears to be one hundred clients, each with one application running. An example of how this departure from fidelity by the prior art simulators would be visible and how it would negatively impact the test that could be achieved is explained below.
Typically, the prior art simulators, asserting that each driving workstation can simulate some number of client workstations, do not physically emulate multiple number of different client workstations, but only one client workstation. This can be verified by using a TCP/IP provided command. TCP/IP provides a simple command: xe2x80x9carp-axe2x80x9d which displays the Address Resolution Protocol (xe2x80x9cARPxe2x80x9d) table. ARP determines through the use of a broadcast message packet the MACID, a physical client address, that corresponds to a specific TCP/IP address, a protocol client address. This pair of addresses is then stored in the ARP table for future use to avoid future network traffic required for the discovery. There will be an entry in the ARP table for each workstation that communicates with a particular TCP/IP host. If this command were to be entered on the system under test, only a single entry would appear, i.e., that of the single workstation communicating with the server. In contrast, if one hundred real clients were present, the ARP table would have one hundred entries, one for each real client.
There are some cases where this distinction is not important. However, if, as an example, there were a bug in the ARP table handling code that only allowed 50 clients to be maintained, then the above test would certify this code as being good up to 100 clients while, once the code was released to customers and actually had to support 100 clients, the bug would become apparent and the code would fail. More insidiously, it would appear that the code had not been tested since the error would appear very quickly in the real environment. Thus, the example test provided by the prior art simulators provides flawed test emulation because it does not provide a realistic testing environment.
Moreover, the test is flawed because the tool used to drive the test does not provide high fidelity to the real client environment. Specifically, the prior art simulator does not exercise all the code paths of a servicing system that the real world servicing system exercises.
An additional disadvantage concerns the network traffic. In the above case, the ARP table is populated with the single entry for a driving workstation. This ARP table population requires a flow of data on the network. The result of this flow yields the ARP table entry. In a real configuration, however, there would have to be multiple flows of network information, one to each of the real clients. Thus, in the prior art simulated environment, ARP would be under represented in the network traffic flows. This would be doubly true as ARP tables have a specific size and entries tend to fall out of this table as they age. In the prior art simulated scenario, this would never occur as traffic for multiple clients is flowing to the single real workstation. In a real environment, this would happen all the time. Thus, once again, the network load simulated under the prior art simulation is different than it is in production. This, once again, leaves code paths that are either not tested at all or that are under tested in the prior art simulated environment and any flaws in these code paths can and will cause failures in production that will not show up in test.
The scenarios where the above-described failures occur can be generalized. Specifically, if the code being tested incurs any per-client costs and/or maintains state information on a per-client basis, then high fidelity testing is required if the code paths that support those per-client items are to be sufficiently tested. FIG. 9 illustrates this concept. The prior art simulators operate in area 1902 providing a small number of clients each of which produces a high bandwidth. Real clients, on the other hand, operate in the mirror reverse. Real clients operate in area 2904, i.e., a large number of clients each of which produces a relatively small bandwidth of data. Thus, while a simulator in the prior art provides the total aggregate bandwidth required by the system under test, the type of this bandwidth is incorrect in that it does not represent a correct number of clients. Testing which has this flaw will show the characteristic that in-lab testing will complete without error but, upon experiencing load in a production environment, the server will fail.
Generally, the above-described behavior is prominent in the services that are provided on the Internet today. Referring back to FIG. 9, if the number of clients is small, areas 1902 and 2904 overlap as shown at 906. Thus, either sort of test may be sufficient. However, as the total bandwidth grows, it is extremely important to ensure that the testing adjusts to the growth correctly. As an example, it is clear that if a company production level doubles in size it is unrealistic to expect that each worker suddenly doubles their workload. Instead, it is more reasonable to expect that the number of workers doubles. Thus, growth typically occurs along the xe2x80x9cXxe2x80x9d axis 908 of the graph in FIG. 9 rather than along the xe2x80x9cYxe2x80x9d axis 910. That is, as Internet usage grows, areas 1902 and 2904 become increasingly distant from one another. This distance represents potential failures that cannot be detected by the existing prior art simulators.
The prior art simulators generally run on the machine types that they are simulating. i.e. if an Intel 80xc3x9786 machine is being simulated, the simulation generally runs on an 80xc3x9786 machine. If an IBM S/390 machine is being simulated, the simulator generally runs on a S/390 machine. This is true because current art simulators depend upon either the applications being run in the simulated platform and/or on the client protocol stack of the simulated platform.
Considering the foregoing, it is highly desirable to have a network simulator that is able to emulate more realistic client/server network traffic and load for testing a system under test. Particularly, it is highly desirable to have a network simulation system that is able to emulate realistically large number of clients. It is also highly desirable to have this simulation system to run on any computer system, independent of specific computer architecture.
Problems arise in simulating multiple clients from a single entity utilizing one network card because the physical LAN entities are designed for a single client, and therefore, have hardware addresses unique to each LAN card which are used for addressing and identifying the single client. Typically these hardware addresses are designated by a unique medium access control identifier (xe2x80x9cMACIDxe2x80x9d) assigned to each LAN card by its manufacturer. In network computing, the MACID identifies the specific client that originates the traffic so that the server contacted can reply and effect a two-way conversation with the client. Thus, it is also highly desirable to have a simulator be able to emulate multiple clients originating traffic while only utilizing a single physical network card. Moreover, it is highly desirable to have this simulator emulating multiple clients to also be able to emanate and absorb traffic as individual clients.
The solutions available today either simulate at a different level, commonly level 4, the session level and/or are specific to a particular protocol, e.g., web server tester, an ODBC tester, etc. These tools yield an imperfect simulation, i.e., provide a few very fast clients rather than many slower clients, and are not extensible when new protocols are developed. There exists a real distinction between system activities generated by few/fast clients and those generated by many/slow clients. Moreover, the current simulation tools invariably require xe2x80x9cPC farmsxe2x80x9d to support the simulation. Thus, what is needed is an improved simulation tool that is able to simulate many and slow clients as well as the few and fast client workstations in order to execute a complete and thorough testing of the system or server resources that are under test.
To be robust, a simulation tool needs to be general purpose in an Intelligent Workstation Environment (xe2x80x9cIWSxe2x80x9d). This general purpose simulation tool must be scriptable and able to check responses. In addition, this tool must have the ability to send simulated requests. The simulation tool must also be centrally and dynamically controllable, and provide a central log of all failures with failure environment. The tool must simulate clients on the order of tens of thousands. Furthermore, the simulated clients must be indistinguishable from real clients. The tool also must be scalable with target processor size. The tool needs to be able to support multiple protocol stacks, and multiple applications per protocol stack. The tool must be able to support multiple delivery vehicles such as LAN, WAN, FDDI types. The tool also must be flexible and extensible. Presently, simulation tools of prior art are unable to support all the above requirements for a robust simulation tool. Therefore, it is highly desirable to have a simulation tool that meets the above-described criteria for a robust simulation tool.
It is an object of the present invention to provide a generalized simulation tool, which is capable of simulating multiple clients at layer 2 of the OSI model in a client/server environment.
It is a further object of the present invention to provide a basic simulation tool capable of simulating multiple clients at layer 2 of the OSI model in a client/server environment.
It is another object of the present invention to provide client/server simulation tool employing a generalized LAN frame generator for simulating situations where the serving entity is a finite state machine, and a basic LAN frame generator where the serving entity is relatively stateless.
The present invention provides a general purpose simulator tool that can be run on a central site, for example, S/390, and which is scalable with the systems under test, for example, other S/390s. The simulator of the present invention builds data frames that emulate frames from multiple clients, thus effectively simulating clients that are indistinguishable from real clients.
The simulator of the present invention is enabled to support multiple protocol stacks or suites, that is, grouping of protocols at multiple layers, including TCP/IP and IPX/SPX. Further yet, the present invention provides a simulator that supports multiple applications per protocol stack.
The present invention also provides a simulator that supports multiple delivery vehicles, thus enabling compatibility with different LAN types as well as other communication connection devices. Further yet, the present invention provides a general purpose simulator that is flexible and extensible.
According to the objects of the present invention, there is provided a simulator tool in a client/server environment for generating one or more LAN frames for enabling multiple client simulation at level 2 of the OSI model. The simulator includes an embedded protocol stack, which allows the manipulation of the simulated frames and allows the tool to respond to dynamic stack-related events in accordance with any level 2 LAN communications protocol. The simulator also includes one or more application programming interfaces (xe2x80x9cAPIsxe2x80x9d) for interaction between the various components of the tool and the protocol stack. The APIs allow the protocol stack to be replaced dynamically and/or to allow multiple simultaneous protocol stacks to be simulated dynamically. The APIs also provide for future compatibility and thus extensibility of the simulation tool with any new protocol stacks which may be implemented in the future, without having to reinvent the bulk of the tool and/or the inclusion of new protocol stacks as the market changes.
Further yet, in the present invention there is provided a scripting facility, facilitating the impression of a workload upon a system under test without the need for a human operator to laboriously enter different complexions of the workload to be impressed upon the system under test and without the need to rebuild the simulation tool with each different workload.
In the present invention, there is also provided one or more embedded protocol application modules (xe2x80x9cPAMsxe2x80x9d) for allowing the actions of the application, e.g., web browser, to be simulated and allowing the handling of dynamic, application-related events. An API is provided between the tool and the PAM and between the PAM and the protocol stack to allow multiple applications to be simulated concurrently and to allow rapid development of new application support.
Further features and advantages of the present invention as well as the structure and operation of various embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers indicate identical or functionally similar elements.